Protecting networks from cyber attacks and overloading

ABSTRACT

Packets may be received by a packet security gateway. Responsive to a determination that an overload condition has occurred in one or more networks associated with the packet security gateway, a first group of packet filtering rules may be applied to at least some of the packets. Applying the first group of packet filtering rules may include allowing at least a first portion of the packets to continue toward their respective destinations. Responsive to a determination that the overload condition has been mitigated, a second group of packet filtering rules may be applied to at least some of the packets. Applying the second group of packet filtering rules may include allowing at least a second portion of the packets to continue toward their respective destinations.

BACKGROUND

The TCP/IP network protocols (e.g., the Transmission Control Protocol(TCP) and the Internet Protocol (IP)) were designed to build large,resilient, reliable, and robust networks. Such protocols, however, werenot originally designed with security in mind. Subsequent developmentshave extended such protocols to provide for secure communication betweenpeers (e.g., Internet Protocol Security (IPsec)), but the networksthemselves remain vulnerable to attack (e.g., Distributed Denial ofService (DDoS) attacks).

The largest TCP/IP network, the Internet, has become criticalcommunications infrastructure for many of the world's countries, such asthe United States of America (US). The US government, US military, andcritical US commercial interests (e.g., utilities, banks, etc.) havebecome operationally dependent on the Internet as the communicationsmedium supporting distributed applications such as the telephone system,utilities grids, and e-commerce. For the US and many other countries, itis a matter of national security that the Internet, as well as some ofthe distributed applications that the Internet supports, hereaftercalled Internet applications, be available for use by certainorganizations during episodes of extreme loading. Extreme loading, oroverloading, of the Internet occurs when the volume of network trafficexceeds the effective transmission capacity of the network. Overloadingof Internet applications occurs when application servers attached to theInternet (e.g., distributed application servers) cannot handle thevolume of service requests that are delivered to the servers by theInternet. Either of these overload cases may occur during cyber attackslaunched by malicious adversaries or during periods of heavy usage bylegitimate users.

Often for reasons of national security, some organizations need to havethe Internet and certain Internet applications available to them duringoverload events. This type of availability requirement has been imposedon pre-Internet telephony systems by some governments. For example, theUS Government Emergency Telecommunications Service (GETS) ensures thatcertain organizations and personnel have emergency access and priorityprocessing for telephone calls on the Public Switched Telephone Network(PSTN). Because of significant differences in protocols, architecture,organization, and operations between the PSTN and the Internet andInternet applications, the technologies, methods, and systems thatsupport GETS cannot be readily ported to the Internet environment.

Accordingly, there is a critical need for technologies, methods, andsystems that can meet availability requirements for the Internet andInternet applications during overload episodes.

SUMMARY

The following presents a simplified summary in order to provide a basicunderstanding of some aspects of the disclosure. It is neither intendedto identify key or critical elements of the disclosure nor to delineatethe scope of the disclosure. The following summary merely presents someconcepts in a simplified form as a prelude to the detailed descriptionbelow.

The core Internet is composed of many Autonomous System (AS) networks.An AS is defined in Internet Engineering Task Force (IETF) Request forComments (RFC) 1930 as a connected group of one or more IP prefixes runby one or more network operators which has a single and clearly definedrouting policy. An AS may be owned and operated by a commercial business(e.g., an Internet Service Provider (ISP)). An ISP may provide Internetconnectivity to its subscribers, which are often enterprises thatoperate their own networks (e.g., private networks) to which associatedendpoints (e.g., enterprise-affiliated desktop computers, servers,mobile devices, etc.) may be attached. These endpoints may host Internetapplication instances (e.g., web servers, web clients, voice telephony,instant messaging, social networking, etc.). These endpoints may beidentified with Internet addresses that follow the Internet Protocol(IP), i.e., IP addresses. The application instances hosted by a givenendpoint may be identified with ports associated with the givenendpoint. For example, a web server instance may listen for requestssent to port 80 of the endpoint hosting the web server instance.

An ISP may need to provide its subscribers with connectivity orreachability to other endpoints that may not be attached to the ISP'ssubscribers' networks; instead, the other endpoints may be attached tonetworks of subscribers to different ISPs. To provide connectivity orreachability, an ISP may connect its AS networks to the AS networks ofother ISPs. These points-of-connection are commonly called peeringpoints, and ISPs that are directly connected to each other's AS networksare commonly called peers. The ISPs may be sufficiently interconnectedvia peering points such that the Internet allows any endpoint with anInternet IP address to send packets (e.g., via routing) to any otherendpoint with an Internet IP address.

The Internet's open connectivity may be exploited by cyber adversariesto launch attacks (e.g., Denial-of-Service (DoS) attacks) againsttargets. In a DoS attack, network resources (e.g., routers, links,endpoints, servers, etc.) may be flooded with so many illegitimateservice requests that legitimate requests are starved (e.g., thelegitimate requests may be effectively denied service). A DoS attack maybe carried out by a botnet, a large collection of compromised hostswhich are controlled and directed by a central command and control agentto send packets to a target victim. One type of DoS attack, commonlycalled a “bandwidth” attack, may flood the network routers and linksthat are immediately upstream of the target with so much malicioustraffic that the network cannot service (e.g., forward) many legitimatepackets that are being routed to the target. Another type of DoS attack,commonly called an “application-level” DoS attack, may flood anapplication server (e.g., a web server) with so many illegitimateservice requests (e.g., HTTP GET requests for web page downloads) thatthe application server is unable to service many legitimate requests,effectively denying service to legitimate users.

It is generally believed that a determined adversary, such as agovernment that is hostile to another country's government, could launchmassive attacks (e.g., DoS attacks) against another country's Internetinfrastructure that are sufficiently large and intense to effectivelydisable the target country's Internet and Internet applications. Thereis much empirical evidence to support this belief. Some of this evidenceis gleaned from episodes of heavy usage by legitimate users, such as theWeb flood by legitimate users that occurred immediately after the Sep.11, 2001 terrorists attacks on the US. More evidence is gleaned from theattacks launched against US banks and financial institutions beginningin the Fall of 2012, and from attacks launched by the loosely associatedhacktivist group known as “Anonymous.” In both the malicious attackscenario and the legitimate flood scenario (and potentially otheroverload scenarios), for reasons of national security, the Internet andsome Internet applications may need to be available to certainorganizations and personnel.

Aspects of this disclosure may relate to ensuring availability of theInternet and some Internet applications to certain organizations andpersonnel, or users, when the Internet is experiencing overloadconditions. Aspects of this disclosure may also relate to restoration ofavailability of the Internet and some Internet applications toprogressively larger sets of users when the Internet is experiencingoverload conditions. Said progression may terminate when normalavailability is restored to all legitimate users.

In some embodiments, packet filtering devices may be located in theInternet at AS network boundary points, such as peering points andsubscriber access points (e.g., Internet access points). The packetfiltering devices may apply sets of filtering rules or policies, topackets traversing network links of the peering or subscriber points. Ifa packet matches a filter rule, the packet may be allowed to continuetowards its destination or prevented or blocked from continuing towardsits destination (e.g., the packet may be dropped), depending on thepacket handling action specified by the matching rule. Some packetfiltering devices may implement a packet handling action thatrate-limits packets that match the associated rule (e.g., the action mayboth block and allow packets depending on whether or not a ratethreshold has been exceeded).

Packet filtering devices may include network firewalls and router accesscontrol lists. A packet filtering device may be referred to herein as aPacket Security Gateway (PSG).

Packet security gateways may be associated with one or more policymanagement servers. Each packet security gateway may receive a policyfrom a policy management server. A policy management server may instructthe packet security gateway to enforce the policy (e.g., to apply rulesspecified in the policy to packet traffic passing through the packetsecurity gateway). The packet security gateways may receive multiplepolicies from policy management servers. These policies may be storedlocally by the packet security gateways and may not need to betransmitted from policy servers to packet security gateways (e.g.,during overload conditions). Additionally or alternatively, the policyservers and packet security gateways may be interconnected by an“out-of-band” management network, which may be physically separate fromthe Internet infrastructure, and may thus be unaffected by Internetoverload conditions.

When an overload condition is detected, some policy management serversmay direct some packet security gateways to enforce a first set ofpolicies. Policies in this first set may contain rules that block allpackets except for packets associated with protocols and applicationsthat are necessary for the Internet and critical Internet applicationsto operate. These protocols and applications may include, for example,Border Gateway Protocol (BGP), the Domain Name System (DNS), and theNetwork Time Protocol (NTP). When this first set of policies is beingenforced, the packet traffic that caused the overload condition may beblocked from ingressing the Internet at Internet access points, or maybe blocked at peering points. Additionally or alternatively, the packettraffic that caused the overload condition may be rate-limited wheningressing the Internet at Internet access points, or may berate-limited at peering points. While this first set of policies isbeing enforced, ISPs and other network operators may take actions toeliminate or mitigate the sources of packet traffic that caused theoverload condition.

In some embodiments, the policy management servers may direct the packetsecurity gateways to enforce a second set of policies. Policies in thissecond set may contain rules from the first set of policies, and mayalso contain one or more additional rules which may allow packetsbetween some Internet applications being used by some critical users orsystems. For example, in a national emergency situation, firstresponders associated with local, state, and federal governmentorganizations may be allowed to use the Internet for telephone calls,text messages, e-mail, web-based services, etc. While this second set ofpolicies is being enforced, ISPs and other network operators maycontinue to take actions to eliminate or mitigate the sources of packettraffic that caused the overload condition.

In some embodiments, the policy management servers may direct the packetsecurity gateways to enforce a third set of policies. Policies in thisthird set may contain rules from the first set of policies and rulesfrom the second set of policies, and may also contain one or moreadditional rules which may allow packets between one or more additionalcritical organizations, personnel, and applications. While this thirdset of policies is being enforced, ISPs and other network operators maycontinue to take actions to eliminate or mitigate the sources of packettraffic that caused the overload condition.

In some embodiments, a cycle of enforcing sets of policies withprogressively broader scopes of users and applications may be repeateduntil normal operation is restored (e.g., until legitimate users havethe Internet and Internet applications available to them as they didbefore the overload conditions occurred).

Other details and features will be described in the sections thatfollow.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure is pointed out with particularity in the appendedclaims. Features of the disclosure will become more apparent upon areview of this disclosure in its entirety, including the drawing figuresprovided herewith.

Some features herein are illustrated by way of example, and not by wayof limitation, in the figures of the accompanying drawings, in whichlike reference numerals refer to similar elements.

FIG. 1 illustrates an exemplary network environment in which one or moreaspects of the disclosure may be implemented.

FIG. 2 illustrates an exemplary network environment with packet securitygateways located at AS network boundaries such as peering points andsubscriber Internet access points.

FIG. 3 illustrates an exemplary packet filtering policy which may beenforced by a packet security gateway located at a peering point.

FIG. 4 illustrates an exemplary packet filtering policy which may beenforced by a packet security gateway located at an Internet accesspoint.

FIG. 5 illustrates an exemplary packet filtering policy which may beenforced by a packet security gateway, and which may allow certain usersor certain Internet applications to communicate.

FIG. 6 illustrates an exemplary network environment with packet securitygateways located at AS network boundaries, such as peering points andsubscriber Internet access points, of an individual ISP that providesprotections to its subscribers.

FIG. 7 illustrates an exemplary method for protecting a network fromoverload conditions while allowing certain users and Internetapplications to communicate across the network.

FIG. 8 illustrates an exemplary packet security gateway.

DETAILED DESCRIPTION

In the following description of various illustrative embodiments,reference is made to the accompanying drawings, which form a parthereof, and in which is shown, by way of illustration, variousembodiments in which aspects of the disclosure may be practiced. It isto be understood that other embodiments may be utilized, and structuraland functional modifications may be made, without departing from thescope of the present disclosure.

Various connections between elements are discussed in the followingdescription. These connections are general and, unless specifiedotherwise, may be direct or indirect, wired or wireless. In thisrespect, the specification is not intended to be limiting.

FIG. 1 illustrates an exemplary network environment in which one or moreaspects of the disclosure may be implemented. Referring to FIG. 1,network environment 100 may be a TCP/IP network environment (e.g., theInternet).

Network environment 100 may include autonomous system (AS) networks 101,102, 103, 104, 105, 106, 107, and 108. AS networks 101-108 may be ownedor operated by various ISPs. AS networks 101-108 may function as transitnetworks (e.g., they may not have Internet-addressable endpointsattached to them and may therefore not terminate any packet microflowsgenerated by Internet applications). For example, packets that ingressesone or more of AS networks 101-108 may also egresses the AS network.Interconnections between any two AS networks 101-108 may be peeringpoints (e.g., a link between AS network 101 and AS network 107 may be apeering point).

Networks 110, 111, 112, 113, 114, and 115 may be owned or operated byvarious enterprises. One or more of networks 110-115 may or may not bean autonomous system network. One or more of networks 110-115 may not bea transit network and may be a private (non-public) network, and maytherefore not be providing Internet service (e.g., an organizationowning or operating one or more of networks 110-115 may not be an ISP).One or more endpoints (not shown in FIG. 1), such as desktop computers,servers, telephones, etc., may be affiliated with these enterprises andmay be attached to one or more of networks 110-115. Such endpoints mayhost instances of various Internet applications, such as web servers andweb clients (e.g., web browsers), text messaging servers and clients, IPtelephony systems, etc. An owner or operator of one or more of networks110-115 may want to allow endpoints attached to their network to be ableto communicate with other endpoints attached to another of networks110-115. For example, an owner or operator of network 110 may want toallow an endpoint attached to network 110 to communicate with anendpoint attached to network 115, which may be owned or operated by adifferent organization than the organization that owns or operatesnetwork 110. To achieve such inter-network communications betweennetworks 110-115, the owners or operators of networks 110-115 maysubscribe to one or more ISPs for Internet service. An ISP may connectone or more of its networks to a subscriber's network. For example, anISP that owns or operates AS network 103 may connect network 103 withnetwork 112, which may be owned or operated by an organization that hassubscribed to the ISP. Connections between subscriber networks and ISPnetworks, such as the connection between network 112 and network 103,may be Internet access points.

ISPs may install routers that support the Border Gateway Control (BGP)protocol, called BGP routers, at the boundaries of their AS networks. ABGP router may know which IP addresses can be reached from itsinterfaces. Using the BGP protocol, a BGP router may advertise itsreachability information to one or more BGP routers located at theborder of different AS networks. For example, a BGP router may advertiseto other BGP routers that are located at the boundaries of peer ASnetworks. A given BGP router may not communicate with every other BGProuter in the Internet. A BGP router may utilize reachabilityinformation received from other BGP routers to compute a local routingtable. A router's routing table may contain entries that associate an IPaddress with one of the router's network interfaces. When a routerreceives a packet, it may look up the packet's destination IP address inthe routing table, and then forward the packet out the network interfacespecified in the routing table entry. The network interface may itselfbe connected to the network interface (e.g., an inbound networkinterface) of another router, which may repeat the lookup- and forwardprocess. Eventually, the packet may reach its destination endpoint.

Utilization of the BGP protocol may be critical for enabling a network'spacket routing service. In one or more implementations of a BGP router,the BGP protocol may also be used to determine if peer BGP routers arefunctioning, for example, via the use of KEEPALIVE messages. If a BGProuter does not receive a KEEPALIVE response from a peer BGP router(e.g., after a configured timeout period), then the BGP router maydetermine that the peer BGP router is no longer functioning, and maystop forwarding packets to the peer BGP router. Accordingly, for anetwork such as the Internet to provide its packet routing service, BGPprotocol communications between peer BGP routers may need to bemaintained.

Internet applications may represent machine-readable IP addresses ofendpoints (e.g., 173.194.75.103) using human-readable domain names(e.g., www.google.com). When an Internet application instance sendspackets over the Internet to an endpoint, the packets may be required tocontain the IP address of the endpoint in the destination IP addressfield of the packets' IP headers. An Internet application may know thedomain name of a destination endpoint but may not know its IP address.An Internet application instance may issue a request to a Domain NameSystem (DNS) to resolve the domain name into an IP address, and the DNSmay respond to the request with an IP address that corresponds to thedomain name. The DNS may be a collection of servers distributed acrossthe Internet that resolve domain names into IP addresses. The DNS andendpoints using the DNS may use the DNS protocol to inter-communicate.Although the Internet may not require the DNS to provide its packetrouting service, and although in theory Internet applications may notneed the DNS to intercommunicate, in practice the DNS may be critical tothe function and operation of many Internet applications. Thus, forInternet applications to function, DNS protocol communications betweenthe DNS and Internet applications may need to be maintained.

The Network Time Protocol (NTP) is a protocol for clock synchronizationbetween computer systems attached to a TCP/IP network (e.g., theInternet). NTP may be architecturally similar to DNS in that there maybe a hierarchical collection of clocks and associated time serversdistributed across the Internet that computer systems may access.Internet applications may depend on synchronized time in order tofunction correctly; thus NTP protocol communications between timeservers and Internet applications may need to be maintained.

There may be other systems and protocols associated with a network thatmay need to be functional or effectively communicating in order for thenetwork or one or more critical network applications to functioncorrectly.

Overload conditions may occur in a network (e.g., the Internet) when anyof several scenarios occur. One scenario may be when many legitimateusers, who may be distributed widely across the network, requestservices (e.g., web page downloads) from the same resource (e.g., a webapplication server) or from a set of resources that are attached to thesame subnet. For example, many legitimate users executing Internetapplication clients (e.g., web browsers) hosted by endpoints attached tonetworks 110-114 may request service from an Internet application server(e.g., a web application server) attached to network 115, during thesame small time window. As the packets containing the requests traversethe Internet and converge on network 115 or the destination Internetapplication server, the volume of aggregate packet traffic may exceedthe capacity of one or more network elements (e.g., routers, switches,network links, gateways, etc.) that are located close to, or immediatelyupstream from, the Internet application server. Finite packet queuescontained in the various network elements may overflow, causing packetsto be dropped. Accordingly one or more requests contained in the droppedpackets may not be serviced by the Internet application server (e.g.,the requesting users and applications may be denied service because ofthe overload condition).

It may also be the case that even if the incoming requests do not causean overload condition, the volume of packets containing responses to therequests may cause an overload condition, for example, in the networkelements located immediately downstream from the Internet applicationserver. For example, this scenario may occur when the Internetapplication is asymmetric (e.g., when the average size, measured inbytes, of responses exceeds the average size of requests). Even thoughall of the requests may have been properly serviced by the Internetapplication server, some of the packets containing responses may havebeen dropped; thus, from the perspective of the service requestors,service may be denied because they may never receive responses to theirrequests.

In another scenario, the volume of requests may not cause an overloadcondition to occur in the network elements immediately upstream from theInternet application server; however, the Internet application servermay not have the processing capacity to service all of the requests. Forexample, if the instantaneous rate of incoming requests exceeds theservice rate of an Internet application server, the requests may bequeued. If the state-of-excess is sustained for a sufficient duration oftime, then the request queue may overflow, causing some requests to bedropped, thereby denying service to the users who issued the droppedrequests.

Overload conditions may also be caused by one or more malicious agents.An overload condition that is caused by malicious agents may be a DoSattack. In a DoS attack, a logical network, or botnet, of maliciousagents, or bots, may generate attack packet traffic when a so-calledcommand-and-control agent directs the bots to launch an attack. Botnetsmay be created when an adversary is able to infect many endpointsdistributed across the Internet with malware that implements the bot.Botnets may be composed of hundreds, thousands, or even millions of botsthat have been identified on the Internet.

The network architecture of a DoS attack may be similar to the networkarchitecture of an overload condition caused by legitimate users. Forexample, a botnet's bots may be hosted by one or more endpoints attachedto networks 110-114. Upon direction from the botnet'scommand-and-control agent, the bots may send many service requests to anInternet application server attached to network 115. These maliciousservice requests or their associated responses may exceed the capacityof the network elements immediately upstream or downstream from theInternet application server, or the malicious service requests mayexceed the capacity of the Internet application server. Accordingly,some legitimate users may be denied service.

Regardless of the cause of an overload condition, some users may requirethe Internet or one or more Internet applications be available for theiruse during the overload condition (e.g., that the services provided bythe Internet or Internet application(s) not be denied to them). Oneapproach to meeting this requirement may be to prevent packets fromnon-required users, agents, endpoints, and Internet applications fromtraversing the Internet and reaching their respective destinations,while simultaneously allowing packets from required users, agents,endpoints, and Internet applications to traverse the Internet and reachtheir respective destinations. In one embodiment such an approach mayutilize one or more packet security gateways to discriminate betweenpackets that should be allowed and packets that should be blocked.

FIG. 2 illustrates an exemplary network environment with packet securitygateways located at AS network boundaries such as peering points andsubscriber Internet access points. Referring to FIG. 2, packet securitygateways (PSGs) 200-220 may have been deployed in network environment100 for the purpose of filtering required and non-required packets insuch a way that during overload conditions, services may not be deniedto certain users, agents, endpoints, or Internet applications. Thepacket security gateways me be located at the boundary points of ASnetworks 101-108 and subscriber networks 110-115 (e.g., at peeringpoints and Internet access points). During an overload condition, one ormore of packet security gateways 200-220 may enforce one or morepolicies (e.g., collections of packet filtering rules), which maydetermine which packet traffic is blocked and which packet traffic isallowed. The policies enforced by the packet security gateways may bechanged over time in order to change the determination of which packettraffic is blocked and which packet traffic is allowed. For example,near the beginning of an overload condition, the scope of packet trafficbeing blocked or allowed, may be broad or narrow, respectively, in orderto ensure that much of the traffic causing the overload condition isblocked, or to ensure that required communications are allowed and fullysupported by the Internet or one or more associated Internetapplications. Over time, as the sources of traffic causing overloadconditions are identified and mitigated, or possibly decontaminated frommalware applications such as bots, the policies may be changed to narrowthe scope of packet traffic being blocked, or to broaden the scope ofpacket traffic being allowed.

When an overload condition is detected, a first set of policies may beenforced by packet security gateways 200-220 to mitigate the overloadcondition and ensure that some users, endpoints, or Internetapplications are able to inter-communicate via network environment 100.Regardless of which users′, endpoints′, or Internet applications'Internet communications are supported by this first set of policies,there may be critical communications between network elements andsystems that may need to be supported in order for the Internet orInternet applications to function properly. These criticalcommunications may be allowed in the first set of policies and in allsubsequent sets of policies. For example, these communications mayinclude one or more of: BGP communications between peer BGP routerslocated at boundary points of ISP-operated AS networks and somesubscriber networks; DNS protocol communications between Internetapplications and DNS servers distributed across the Internet; and NTPcommunications between Internet elements, applications, or time serversdistributed across the Internet. Additionally or alternatively, theremay be other protocols that are considered critical; accordingly, afirst set of policies may also support communications for these otherprotocols.

FIG. 3 illustrates an exemplary packet filtering policy which may beenforced by a packet security gateway located at a peering point.Referring to FIG. 3, policy 300 may contain one or more filtering rulerepresentations. For example, packet security gateways may filter onfive (5) fields in an IP packet: source and destination IP addressfields, source and destination port fields (e.g., those contained in theencapsulated transport protocol packet, if any), and protocol (for IPversion 4, as shown) or next header (for IP version 6, not shown). Thefive fields may be referred to as a “5-tuple”. 5-tuple filtering rulesmay specify values for any number of the five fields (e.g., a filteringrule may only filter packets on a single field such as source IPaddress, or a filtering rule may filter on any combination of two,three, or four fields, or all five fields). Each rule may be associatedwith a packet handling action, which may be, for example, BLOCK (e.g.,drop the packet) or ALLOW (e.g., forward the packet towards itsdestination).

The rules in policy 300 may allow certain BGP protocol communications,certain DNS protocol communications, and certain NTP protocolcommunications. Policy 300 may, for example, be enforced by a packetsecurity gateway located at a peering point between two transitnetworks. For example, packet security gateway 220 may be located at apeering point between AS network 107 and AS network 108. A BGP router(not illustrated) may be located at each end of a network linkconnecting AS network 107 and AS network 108. An owner or operator of ASnetwork 107 may assign IP version 4 address 123.56.89.0 to a networkinterface on the BGP router at the boundary of AS network 107, and anowner or operator of AS network 108 may assign IP version 4 address87.65.21.0 to a network interface on the BGP router at the boundary ofAS network 108. A network link may connect interface 123.56.89.0 tonetwork interface 87.65.21.0. This network link may pass through packetsecurity gateway 220, but as the network interfaces of packet securitygateway 220 may not have IP addresses assigned to them, at the IP level,packet security gateway 220 may be transparent to the BGP routers.

Rule 1 301 of policy 300 may allow BGP packets sent by a BGP client fromthe network interface 123.56.89.0 and from any source port (as denotedby the “*” wildcard symbol) to network interface 87.65.21.0 and port179, (e.g., a port associated with a BGP listener or BGP server). Rule 2302 may allow BGP packets to be sent by a BGP client from the networkinterface 87.65.21.0 and from any source port to network interface123.56.89.0 and port 179. Rule 3 303 and rule 4 304 may respectivelyallow packets containing responses to any requests or messages containedin packets allowed by rule 2 302 or rule 1 301 to be sent back to theirrequestors. BGP may use TCP as its transport protocol; accordingly, theprotocol field value in rules 1-4 301-304 may be set to TCP.

Rule 5 305 and rule 6 306 may allow DNS protocol packets to pass throughpacket security gateway 220. Rules 5 305 and 6 306 may not includerestrictions on the source IP addresses and destination IP addresses.For example, because DNS clients and DNS servers may be located insubscriber networks connected to the edge of network environment 100(e.g., networks 110-115) packet filtering rules applied by a packetsecurity gateway located at a peering point between two transit networks(e.g., packet security gateway 220 located between transit networks 107and 108) may not have restrictions on the source and destination IPaddresses of DNS protocol packets (e.g., because potentially any pair ofDNS clients and servers could be communicating through the peeringpoint). Rule 5 305 may allow packets that contain any DNS client'srequest and that are destined for any DNS server, which may be listeningfor requests on one or more ports (e.g., on port 53). Rule 6 306 mayallow packets that contain DNS server responses to any requestscontained in the packets allowed by rule 5 305. The DNS protocol may betransported using either TCP or the User Datagram Protocol (UDP);accordingly, the Protocol field in rule 5 305 and rule 6 306 may allowany value.

Rule 7 307 and rule 8 308 may allow NTP protocol packets to pass throughpacket security gateway 220. Similar to DNS, NTP clients and NTP serversmay be located in subscriber networks connected to the edge of networkenvironment 100 (e.g., networks 110-115); thus, packet filtering rulesapplied by a packet security gateway located at a peering point betweentwo transit networks (e.g., packet security gateway 220 located betweentransit networks 107 and 108) may not have restrictions on the sourceand destination IP addresses of NTP protocol packets because potentiallyany pair of NTP clients and servers could be communicating through thepeering point. Rule 7 307 may allow packets that contain any NTPclient's request and that are destined for any NTP server, which may belistening for requests on one or more ports (e.g., 123). Rule 8 308 mayallow packets that contain NTP server responses to any requestscontained in the packets allowed by rule 7 307. NTP may use UDP as itstransport protocol; accordingly, the Protocol field in rule 7 307 andrule 8 308 may be set to UDP.

Rule 9 309 may block any packet that does not match any of rules 1-8301-308. For example, packet security gateway 220 may apply rules topackets in the order in which they appear in the policy that containsthem. Accordingly, rule 9 309 may blocks packets that that do not matchany of rules 1-8 301-308 (e.g., one or more packets associated with thecreation of an overload condition).

Policy 300 may be enforced by one or more packet security gateways atany peering point or Internet access point in network environment 100.In some embodiments, more restrictive rules may be contained in policiesenforced by packet security gateways located near the edge of networkenvironment 100 (e.g., at Internet access points). For example, tomitigate or even eliminate overload conditions at locations near theedge. In one type of DoS attack, known as an open DNS resolver attack, abotnet may cause many DNS servers to send packets to a target resource(e.g., a subscriber network's Internet access points or a company'spublic e-commerce web server) located at or near the edge of theInternet. Rule 5 305 and rule 6 306 of policy 300 may not block suchpackets. At an Internet access point, however, the IP addresses of theDNS clients and servers that are expected to be communicating across theInternet access point may be known to the operators of either thesubscriber network or the ISP network connected by the Internet accesspoint. Packet filtering rules that filter DNS protocol packets and thatspecify specific IP addresses of DNS endpoints in their source anddestination IP address fields, may be enforced by packet securitygateways located at Internet access points and may block most or all ofthe packets generated by an open DNS resolver attack, thereby mitigatingor eliminating any overload conditions caused by such an attack.

FIG. 4 illustrates an exemplary packet filtering policy which may beenforced by a packet security gateway located at an Internet accesspoint. Referencing FIG. 4, rules 10 401 and 11 402 may be contained inpolicy 400. Policy 400 may be enforced by packet security gateway 200,which may be located at an Internet access point between subscribernetwork 110 and AS network 102. Subscriber network 110 may have beenallocated IP version 4 addresses with subnet prefix 32.10.87.0/24. DNSclients attached to network 110 may have all of their DNS requestsrouted to a DNS server with IP address 13.57.92.46, which may beexternal to network 110, and which may be considered to be trusted bythe operators of network 110. Rule 10 401 may allow packets containingrequests from DNS clients attached to network 110 and destined for port53 on DNS server 13.57.92.46. Rule 11 402 may allow packets containingresponses from DNS server 13.57.92.46 and destined for one or more DNSclients attached to network 110. Rule 12 403 may block any DNS serverpackets destined for network 110, as such packets may be part of an openDNS resolver attack, or may otherwise be packets from a DNS server thatwere not requested by a DNS client attached to network 110. In someembodiments, rule 12 403 may not be included in policy 400. For example,the last rule in the policy 400 may be a block rule like rule 9 309 inpolicy 300.

An overload condition may be highly mitigated or even eliminated byhaving packet security gateways 200-220 in network environment 100enforce a first set of policies which is composed of policies similar topolicy 300 and policy 400. This first set of policies may, however, alsoprevent one or more legitimate users or their Internet applications fromcommunicating across network environment 100. For example, overloadconditions may occur when there is a large DoS attack or many DoSattacks. Overload conditions may also occur when there is a widespreademergency condition that causes many legitimate users to attempt toaccess the same resources (e.g., a telephony system or news web site).While this first set of policies is being enforced, network operatorsmay take actions to mitigate or eliminate the sources of packets thatcaused the original overload conditions. For example, network operatorsmay prevent endpoints suspected of hosting bots from accessing theInternet or network operators may severely rate-limit some types oftraffic that are believed to be causing the overload conditions.

It may be desirable or may be required by local laws or regulations thatsome users (e.g., first responders) be guaranteed services from theInternet or from certain Internet applications, despite the overloadconditions. To provide such guarantees, a second set of policies may beenforced by one or more of packet security gateways 200-220 in networkenvironment 100. These policies may contain all of the rules containedin the first set of policies and one or more additional rules that allowcertain users (e.g., first responders) or certain Internet applicationsto communicate over network environment 100.

For example, all users with endpoints attached to network 110 and allusers with endpoints attached to network 112 may be allowed tocommunicate, using the HTTP protocol, with web application serversattached to network 113. Network 110 may have been allocated IP version4 addresses with subnet prefix 10.10.87.0/24. Network 112 may have beenallocated IP addresses with subnet prefix 12.12.87.0/24, and network 113may have been allocated IP addresses with subnet prefix 13.13.87.0/24.

FIG. 5 illustrates an exemplary packet filtering policy which may beenforced by a packet security gateway, and which may allow certain usersor certain Internet applications to communicate. Referring to FIG. 5,policy 500 may include one or more of the rules from policy 300 orpolicy 400. Policy 500 may also contain rules 13-16 501-504. Rule 13 501may allow packets sourced from HTTP clients (e.g., web browsers)attached to network 110 and destined for one or more HTTP servers (e.g.,one or more web application servers on port 80) attached to network 113.Rule 14 502 may allow packets sourced by the HTTP servers attached tonetwork 113 and destined for endpoints attached to network 110. Suchpackets may, for example, contain responses to HTTP requests issued byHTTP clients attached to network 110. Rule 15 503 and rule 16 504 may besimilar to rule 13 501 and rule 14 502 except they may allow packetscontaining HTTP client requests and HTTP server responses betweennetworks 112 and 113.

An overload condition may be highly mitigated or even eliminated, andcertain users or certain Internet applications may be allowed tocommunicate over network environment 100, by having packet securitygateways 200-220 in network environment 100 enforce a second set ofpolicies which is composed of policies similar to policy 500. While thissecond set of policies is being enforced, network operators may takeactions to mitigate or eliminate the sources of packets that caused theoriginal overload conditions.

Later, a third set of policies may be enforced by packet securitygateways 200-220 in network environment 100 which may contain all of therules contained in the second set of policies (which may themselves havecontained all of the rules contained in the first set of policies) andmay also contain one or more additional rules that allow more usersand/or more Internet applications to communicate over networkenvironment 100. While the third set of policies is being enforced,network operators may take further actions to mitigate or eliminatesources of packets that caused the overload conditions. Later, a fourthset of policies may be enforced that incorporates the third set ofpolicies and broadens the scope of user and/or Internet applicationsthat may communicate over network environment 100. Such a cycle may berepeated until the normal operation of one or more of networkenvironment 100, its users, or its Internet applications, is restored,or the sources of traffic which caused the original overload conditionsare sufficiently mitigated or eliminated such that users and Internetapplications are not denied service because of overload conditions.

In some embodiments, packet security gateways may be required to belocated at all peering points or Internet access points in networkenvironment 100. In other embodiments, this practice may be relaxedwhile still providing protection from overload conditions and whilestill providing some users and Internet applications with communicationsservices. For example, an individual ISP may be able to offer protectionfrom overload conditions and still support selected communications forits subscribers.

FIG. 6 illustrates an exemplary network environment with packet securitygateways located at AS network boundaries, such as peering points andsubscriber Internet access points, of an individual ISP that providesprotections to its subscribers. Referring to FIG. 6, an ISP (e.g.,SecureISP) may own or operate AS networks 102, 103, and 106 in networkenvironment 100. SecureISP may have located packet security gateways(e.g., packet security gateways 200-207, 210, 213, 214, and 215) at allthe peering points and Internet access points of its networks. One ormore other ISPs that own or operate AS networks 101, 104, 105, 107, and108 may not have installed packet security gateways at peering pointsand Internet access points of their networks.

An overload condition may occur in network 113, which may be owned oroperated by a subscriber to SecureISP. By enforcing one or more policiessimilar to policy 300 at its peering points and by enforcing policiessimilar to policy 400 at its Internet access points, SecureISP mayeliminate or highly mitigate the overload condition in network 113. Forexample, regardless of the source of the packet traffic that caused theoverload condition (e.g., any combination of endpoints attached tonetworks 110, 111, 112, 114, and 115), the traffic may be filtered by apolicy included in the first set of policies because the traffic may berequired to attempt to pass through one of the packet security gatewaysoperated by SecureISP while being routed towards network 113. While thefirst set of policies is being enforced, SecureISP may take actions tomitigate or eliminate one or more sources of the traffic causing theoverload condition. For example, SecureISP may take actions to mitigateor eliminate one or more sources of traffic that are attached to itssubscribers' networks.

Later, after enforcing the first set of policies, SecureISP may want toallow all users with endpoints attached to its subscriber's network 110and all users with endpoints attached to its subscriber's network 112 tocommunicate, using the HTTP protocol, with web application serversattached to its subscriber's network 113. Network 110 may have beenallocated IP version 4 addresses with subnet prefix 10.10.87.0/24.Network 112 may have been allocated IP addresses with subnet prefix12.12.87.0/24. Network 113 may have been allocated IP addresses withsubnet prefix 13.13.87.0/24. By enforcing a second set of policiessimilar to policy 500 at its peering points and its Internet accesspoints, SecureISP may eliminate or highly mitigate the overloadcondition in network 113 while allowing HTTP clients (e.g., webbrowsers) attached to its subscribers' networks 110 and 112 tocommunicate with HTTP servers (e.g., web application servers) attachedto its subscriber's network 113.

Depending on the routing polices being used in network environment 100,packet traffic generated by HTTP clients and HTTP servers attached tonetworks 110, 112, and 113 may be required to traverse one or more of ASnetworks 101, 104, 105, 107, and 108, which may not have packet securitygateways located at their peering points and Internet access points.Packet traffic generated by HTTP clients and HTTP servers attached tonetworks 110, 112, and 113 may traverse AS networks which may also betransporting traffic that may be causing overload conditions at varioussubscriber networks 110-115. Given the architecture, operation, andbehavior of network environment 100, it may be unlikely that any one ormore of AS networks 101, 104, 105, 107, and 108 are themselvesexperiencing overload conditions that may disrupt communications betweenHTTP clients and HTTP servers attached to networks 110, 112, and 113.Accordingly, SecureISP may be able to offer effective protections fromoverload conditions to its subscribers, even though other ISPs may notoffer similar protections and may transport some or most of the trafficthat may be causing overload conditions in SecureISP's subcribers'networks.

FIG. 7 illustrates an exemplary method for protecting a network fromoverload conditions while allowing certain users and Internetapplications to communicate across the network. Referring to FIG. 7, atstep 702, packets may be received. For example, packet security gateway200 may receive packets from network 110. At step 704, responsive to adetermination that an overload condition has occurred, a first group ofpacket filtering rules may be applied to at least some of the packets.For example, an overload condition may occur in network 113, andresponsive to a determination that the overload condition in network 113has occurred, packet security gateway 200 may apply one or more of rules1-9 301-309 of policy 300 to at least some of the packets received fromnetwork 110. At step 706, responsive to a determination that theoverload condition has been mitigated, a second group of packetfiltering rules may be applied to at least some of the packets. Forexample, responsive to a determination that the overload condition innetwork 113 has been mitigated, packet security gateway 200 may applyone of more of rules 13-16 501-504 to at least some of the packetsreceived from network 110.

FIG. 8 illustrates an exemplary packet security gateway. Referring toFIG. 8, as indicated above, packet security gateway 220 may be locatedbetween AS networks 107 and 108. For example, packet security gateway220 may be located at network boundary 802. Packet security gateway 220may include one or more processors 804, memory 806, network interfaces808 and 810, packet filter 812, and management interface 814.Processor(s) 804, memory 806, network interfaces 808 and 810, packetfilter 812, and management interface 814 may be interconnected via databus 816. Network interface 808 may connect packet security gateway 220to AS network 107. Similarly, network interface 810 may connect packetsecurity gateway 220 to AS network 108. Memory 806 may include one ormore program modules that when executed by processor(s) 804, mayconfigure packet security gateway 220 to perform one or more of variousfunctions described herein.

Packet security gateway 220 may be configured to receive a policy (e.g.,one or more of policies 300, 400, or 500) from one or more securitypolicy management servers (not illustrated). For example, packetsecurity gateway 220 may receive policy 818 from a security policymanagement server via management interface 814 (e.g., via out-of-bandsignaling) or network interface 808 (e.g., via in-band signaling).Packet security gateway 220 may include one or more packet filters orpacket discriminators, or logic for implementing one or more packetfilters or packet discriminators. For example, packet security gateway220 may include packet filter 812, which may be configured to examineinformation associated with packets received by packet security gateway220 and forward such packets to one or more of operators 820, 822, or824 based on the examined information. For example, packet filter 812may examine information associated with packets received by packetsecurity gateway 220 (e.g., packets received from AS network 107 vianetwork interface 808) and forward the packets to one or more ofoperators 820, 822, or 824 based on the examined information.

Policy 818 may include one or more rules and the configuration of packetfilter 812 may be based on one or more of the rules included in policy818. For example, policy 818 may include one or more rules specifyingthat packets having specified information should be forwarded tooperator 820, that packets having different specified information shouldbe forwarded to operator 822, and that all other packets should beforwarded to operator 824. Operators 820, 822, and 824 may be configuredto perform one or more functions on packets they receive from packetfilter 812. For example, one or more of operators 820, 822, or 824 maybe configured to forward packets received from packet filter 812 into ASnetwork 108, forward packets received from packet filter 812 to an IPsecstack (not illustrated) having an IPsec security associationcorresponding to the packets, or drop packets received from packetfilter 812. In some embodiments, one or more of operators 820, 822, or824 may be configured to drop packets by sending the packets to a local“infinite sink” (e.g., the /dev/null device file in a UNIX/LINUXsystem).

The functions and steps described herein may be embodied incomputer-usable data or computer-executable instructions, such as in oneor more program modules, executed by one or more computers or otherdevices to perform one or more functions described herein. Generally,program modules include routines, programs, objects, components, datastructures, etc. that perform particular tasks or implement particularabstract data types when executed by one or more processors in acomputer or other data processing device. The computer-executableinstructions may be stored on a computer-readable medium such as a harddisk, optical disk, removable storage media, solid-state memory, RAM,etc. As will be appreciated, the functionality of the program modulesmay be combined or distributed as desired in various embodiments. Inaddition, the functionality may be embodied in whole or in part infirmware or hardware equivalents, such as integrated circuits,application-specific integrated circuits (ASICs), field programmablegate arrays (FPGA), and the like. Particular data structures may be usedto more effectively implement one or more aspects of the disclosure, andsuch data structures are contemplated to be within the scope of computerexecutable instructions and computer-usable data described herein.

Although not required, one of ordinary skill in the art will appreciatethat various aspects described herein may be embodied as a method, anapparatus, or as one or more computer-readable media storingcomputer-executable instructions. Accordingly, those aspects may takethe form of an entirely hardware embodiment, an entirely softwareembodiment, an entirely firmware embodiment, or an embodiment combiningsoftware, hardware, and firmware aspects in any combination.

As described herein, the various methods and acts may be operativeacross one or more computing servers and one or more networks. Thefunctionality may be distributed in any manner, or may be located in asingle computing device (e.g., a server, a client computer, etc.).

Aspects of the disclosure have been described in terms of illustrativeembodiments thereof. Numerous other embodiments, modifications, andvariations within the scope and spirit of the appended claims will occurto persons of ordinary skill in the art from a review of thisdisclosure. For example, one of ordinary skill in the art willappreciate that the steps illustrated in the illustrative figures may beperformed in other than the recited order, and that one or more stepsillustrated may be optional.

What is claimed is:
 1. A method, comprising: at a computing platformcomprising at least one processor, memory, and communication interface:receiving, via the communication interface, a plurality of packets;responsive to a determination that an overload condition has occurred inone or more networks associated with the computing platform, applying,by the processor and to at least some of the plurality of packets, afirst group of packet filtering rules that comprises at least onefive-tuple indicating a first set of packets that should be allowed tocontinue toward their respective destinations, wherein applying thefirst group of packet filtering rules includes allowing at least a firstportion of the plurality of packets that fall within the first set ofpackets to continue toward their respective destinations; and responsiveto a determination that the overload condition has been mitigated to afirst degree, applying, to at least some of the plurality of packets, asecond group of packet filtering rules that comprises at least onefive-tuple indicating a second set of packets that should be allowed tocontinue toward their respective destinations, wherein applying thesecond group of packet filtering rules includes allowing at least asecond portion of the plurality of packets that fall within the secondset of packets to continue toward their respective destinations, andwherein the second set of packets includes more packets than the firstset of packets.
 2. The method of claim 1, comprising: responsive to adetermination that the overload condition has been mitigated to a seconddegree, applying, to at least some of the plurality of packets, a thirdgroup of packet filtering rules that comprises at least one five-tupleindicating a third set of packets that should be allowed to continuetoward their respective destinations, wherein applying the third groupof packet filtering rules includes allowing at least a third portion ofthe plurality of packets that fall within the third set of packets tocontinue toward their respective destinations, wherein the third set ofpackets includes more packets than the second set of packets, andwherein the second degree is a greater degree of mitigation than thefirst degree.